{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Canary Deployment with Seldon and Istio"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prerequisites\n",
    "You will need\n",
    " - [Git clone of Seldon Core](https://github.com/SeldonIO/seldon-core)\n",
    " - A running Kubernetes cluster with kubectl authenticated\n",
    " - [seldon-core Python package](https://pypi.org/project/seldon-core/) (```pip install seldon-core>=0.2.6.1```)\n",
    " - [Helm client](https://helm.sh/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Creating a Kubernetes Cluster\n",
    "\n",
    "Follow the [Kubernetes documentation to create a cluster](https://kubernetes.io/docs/setup/).\n",
    "\n",
    "***This demo needs egress when running the load test to allow MNIST digits to be downloaded. If you want to run the load test then you will need to follow the docs on egress [here](https://istio.io/docs/tasks/traffic-management/egress/#calling-external-services-directly) if you run istio in a way that egress is blocked***\n",
    "\n",
    "Once created ensure ```kubectl``` is authenticated against the running cluster."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "namespace/seldon created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl create namespace seldon"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Context \"minikube\" modified.\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl config set-context $(kubectl config current-context) --namespace=seldon"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "clusterrolebinding.rbac.authorization.k8s.io/kube-system-cluster-admin created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl create clusterrolebinding kube-system-cluster-admin --clusterrole=cluster-admin --serviceaccount=kube-system:default"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Install Helm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "serviceaccount/tiller created\n",
      "clusterrolebinding.rbac.authorization.k8s.io/tiller created\n",
      "$HELM_HOME has been configured at /home/clive/.helm.\n",
      "\n",
      "Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster.\n",
      "\n",
      "Please note: by default, Tiller is deployed with an insecure 'allow unauthenticated users' policy.\n",
      "To prevent this, run `helm init` with the --tiller-tls-verify flag.\n",
      "For more information on securing your installation see: https://docs.helm.sh/using_helm/#securing-your-helm-installation\n",
      "Happy Helming!\n"
     ]
    }
   ],
   "source": [
    "!kubectl -n kube-system create sa tiller\n",
    "!kubectl create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=kube-system:tiller\n",
    "!helm init --service-account tiller"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Waiting for deployment \"tiller-deploy\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "deployment \"tiller-deploy\" successfully rolled out\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deploy/tiller-deploy -n kube-system"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup Istio\n",
    "\n",
    "Ensure you have istio installed. Follow their [docs](https://istio.io/docs)\n",
    "\n",
    "For this example we will create the default istio gateway for seldon which needs to be called `seldon-gateway`. You can supply your own gateway by adding to your SeldonDeployments resources the annotation `seldon.io/istio-gateway` with values the name of your istio gateway."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a gateway for our istio-ingress"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "gateway.networking.istio.io/seldon-gateway created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl create -f ../../../notebooks/resources/seldon-gateway.yaml"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Label our namespace so istio creates sidecars"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "namespace/seldon labeled\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl label namespace seldon istio-injection=enabled"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you are using Minikube for your Kubernetes cluster you will need to run as root in a separte terminal:\n",
    "```\n",
    "minikube tunnel\n",
    "```\n",
    "This will allow a LoadBalancer to be simulated on your local machine. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "INGRESS_HOST=!kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}'\n",
    "INGRESS_PORT=!kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name==\"http2\")].port}'\n",
    "ISTIO_GATEWAY=INGRESS_HOST[0]+\":\"+INGRESS_PORT[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check the istio gateway address"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'10.109.199.85:80'"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ISTIO_GATEWAY"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To view the istio traffic you can go to the istio grafana dashboard. In a separate terminal port-forward to it:\n",
    "\n",
    "See their docs [here](https://istio.io/docs/tasks/telemetry/metrics/using-istio-dashboard/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Start seldon-core"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME:   seldon-core\n",
      "LAST DEPLOYED: Sun Jun 30 11:38:18 2019\n",
      "NAMESPACE: seldon-system\n",
      "STATUS: DEPLOYED\n",
      "\n",
      "RESOURCES:\n",
      "==> v1/ClusterRole\n",
      "NAME                          AGE\n",
      "seldon-operator-manager-role  0s\n",
      "\n",
      "==> v1/ClusterRoleBinding\n",
      "NAME                                 AGE\n",
      "seldon-operator-manager-rolebinding  0s\n",
      "\n",
      "==> v1/ConfigMap\n",
      "NAME                     DATA  AGE\n",
      "seldon-spartakus-config  3     1s\n",
      "\n",
      "==> v1/Pod(related)\n",
      "NAME                                         READY  STATUS             RESTARTS  AGE\n",
      "seldon-operator-controller-manager-0         0/1    ContainerCreating  0         0s\n",
      "seldon-spartakus-volunteer-5866b6df59-sslhd  0/1    ContainerCreating  0         0s\n",
      "\n",
      "==> v1/Secret\n",
      "NAME                                   TYPE    DATA  AGE\n",
      "seldon-operator-webhook-server-secret  Opaque  0     1s\n",
      "\n",
      "==> v1/Service\n",
      "NAME                                        TYPE       CLUSTER-IP    EXTERNAL-IP  PORT(S)  AGE\n",
      "seldon-operator-controller-manager-service  ClusterIP  10.96.98.198  <none>       443/TCP  0s\n",
      "\n",
      "==> v1/ServiceAccount\n",
      "NAME                              SECRETS  AGE\n",
      "seldon-core-seldon-core-operator  1        1s\n",
      "seldon-spartakus-volunteer        1        1s\n",
      "\n",
      "==> v1/StatefulSet\n",
      "NAME                                READY  AGE\n",
      "seldon-operator-controller-manager  0/1    0s\n",
      "\n",
      "==> v1beta1/ClusterRole\n",
      "NAME                        AGE\n",
      "seldon-spartakus-volunteer  0s\n",
      "\n",
      "==> v1beta1/ClusterRoleBinding\n",
      "NAME                        AGE\n",
      "seldon-spartakus-volunteer  0s\n",
      "\n",
      "==> v1beta1/CustomResourceDefinition\n",
      "NAME                                         AGE\n",
      "seldondeployments.machinelearning.seldon.io  1s\n",
      "\n",
      "==> v1beta1/Deployment\n",
      "NAME                        READY  UP-TO-DATE  AVAILABLE  AGE\n",
      "seldon-spartakus-volunteer  0/1    1           0          0s\n",
      "\n",
      "\n",
      "NOTES:\n",
      "NOTES: TODO\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!helm install ../../../helm-charts/seldon-core-operator --name seldon-core --set istio.enabled=true --set usageMetrics.enabled=true   --namespace seldon-system"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Waiting for 1 pods to be ready...\n",
      "partitioned roll out complete: 1 new pods have been updated...\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deploy/seldon-controller-manager -n seldon-system"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Serve Single Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "from random import randint,random\n",
    "import json\n",
    "from matplotlib import pyplot as plt\n",
    "import numpy as np\n",
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "from seldon_core.seldon_client import SeldonClient\n",
    "import seldon_core\n",
    "\n",
    "def gen_image(arr):\n",
    "    two_d = (np.reshape(arr, (28, 28)) * 255).astype(np.uint8)\n",
    "    plt.imshow(two_d,cmap=plt.cm.gray_r, interpolation='nearest')\n",
    "    return plt\n",
    "\n",
    "def download_mnist():\n",
    "    return input_data.read_data_sets(\"MNIST_data/\", one_hot = True)\n",
    "\n",
    "def predict_rest_mnist(mnist,deployment_name,namespace,istio_gateway):\n",
    "    sc = SeldonClient(deployment_name=deployment_name,namespace=namespace,gateway_endpoint=istio_gateway)\n",
    "    batch_xs, batch_ys = mnist.train.next_batch(1)\n",
    "    chosen=0\n",
    "    gen_image(batch_xs[chosen]).show()\n",
    "    data = batch_xs[chosen].reshape((1,784))\n",
    "    features = [\"X\"+str(i+1) for i in range (0,784)]  \n",
    "    r = sc.predict(gateway=\"istio\",transport=\"rest\",shape=(1,784),data=data,payload_type='ndarray',names=features)\n",
    "    predictions = seldon_core.utils.seldon_message_to_json(r.response)\n",
    "    print(predictions)\n",
    "    #print(\"Route:\"+json.dumps(predictions[\"meta\"][\"routing\"],indent=2))\n",
    "    fpreds = [ '%.2f' % elem for elem in predictions[\"data\"][\"ndarray\"][0] ]\n",
    "    m = dict(zip(predictions[\"data\"][\"names\"],fpreds))\n",
    "    print(json.dumps(m,indent=2))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting MNIST_data/train-images-idx3-ubyte.gz\n",
      "Extracting MNIST_data/train-labels-idx1-ubyte.gz\n",
      "Extracting MNIST_data/t10k-images-idx3-ubyte.gz\n",
      "Extracting MNIST_data/t10k-labels-idx1-ubyte.gz\n"
     ]
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "from visualizer import get_graph\n",
    "mnist = download_mnist()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io/mnist-classifier created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply -f mnist_v1.json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "deployment \"mnist-deployment-sk-mnist-predictor-73d7608\" successfully rolled out\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deploy/mnist-deployment-sk-mnist-predictor-73d7608"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAADK9JREFUeJzt3W+oXPWdx/HPZ2NCIA3+2cwmwZq9tcqCKKZlCEJlqdZWq4XYB0rzoNxF2RRpIJU8WLXiChKQZW3tg6WQmkuSpWsrtGoeiFsbV6SwKV7/bNS6TWy8IYn5c4PFpqB0k3z3wT2W23jnzDhzzpy5+b5fcLkz53vm/L4M+eTMzO/c+TkiBCCfv2q6AQDNIPxAUoQfSIrwA0kRfiApwg8kRfiBpAg/kBThB5I6b5iDLVu2LMbGxoY5JJDK1NSUTpw44V72HSj8tm+S9ANJCyQ9FhEPl+0/NjamycnJQYYEUKLdbve8b98v+20vkPRvkr4q6QpJ62xf0e/xAAzXIO/510h6OyL2R8SfJP1E0tpq2gJQt0HCf7Gkg7PuHyq2/QXb621P2p6cnp4eYDgAVar90/6I2BIR7Yhot1qtuocD0KNBwn9Y0iWz7n+62AZgHhgk/C9Jutz2Z2wvkvQNSTuraQtA3fqe6ouIU7Y3SPpPzUz1TUTEm5V1BqBWA83zR8Qzkp6pqBcAQ8TlvUBShB9IivADSRF+ICnCDyRF+IGkCD+QFOEHkiL8QFKEH0iK8ANJEX4gKcIPJEX4gaQIP5AU4QeSIvxAUoQfSIrwA0kRfiApwg8kNdQluoFPYuPGjaX13bt3l9Y3b97csXbDDTf01dO5hDM/kBThB5Ii/EBShB9IivADSRF+ICnCDyQ10Dy/7SlJJyWdlnQqItpVNIUcTp8+XVp/9tlnS+t79+4trT/11FMda8zzV3ORz3URcaKC4wAYIl72A0kNGv6Q9AvbL9teX0VDAIZj0Jf910bEYdt/I+k52/8bES/O3qH4T2G9JK1atWrA4QBUZaAzf0QcLn4fl/SkpDVz7LMlItoR0W61WoMMB6BCfYff9hLbSz+6Lekrkt6oqjEA9RrkZf9ySU/a/ug4/xER5XMzAEZG3+GPiP2Srq6wFyTz2GOPlda7zeN3s3DhwoEef65jqg9IivADSRF+ICnCDyRF+IGkCD+QFF/djVp98MEHHWtPPPHEEDvB2TjzA0kRfiApwg8kRfiBpAg/kBThB5Ii/EBSzPOjVo8++mjH2vPPP1/r2DfeeGOtx5/vOPMDSRF+ICnCDyRF+IGkCD+QFOEHkiL8QFLM82MgZ86cKa3v3r27trGvueaa0vr1119f29jnAs78QFKEH0iK8ANJEX4gKcIPJEX4gaQIP5BU13l+2xOSvibpeERcWWy7SNJPJY1JmpJ0e0T8vr42Mao2b95cWt+5c2dtY3f7PoBFixbVNva5oJcz/zZJN5217R5JuyLickm7ivsA5pGu4Y+IFyW9d9bmtZK2F7e3S7q14r4A1Kzf9/zLI+JIcfuopOUV9QNgSAb+wC8iQlJ0qtteb3vS9uT09PSgwwGoSL/hP2Z7pSQVv4932jEitkREOyLarVarz+EAVK3f8O+UNF7cHpf0dDXtABiWruG3/bik/5b0d7YP2b5T0sOSvmx7n6QbivsA5pGu8/wRsa5D6UsV94IRtHfv3tL6xMREbWNv27attL548eLaxs6AK/yApAg/kBThB5Ii/EBShB9IivADSfHV3Sh1yy23lNanpqb6PvaGDRtK6+Pj46V1DIYzP5AU4QeSIvxAUoQfSIrwA0kRfiApwg8kxTx/co888khpff/+/QMdv+zPbu++++6Bjo3BcOYHkiL8QFKEH0iK8ANJEX4gKcIPJEX4gaSY5z/H7dmzp7R+//33l9bPnDkz0PgPPPBAx9qll1460LExGM78QFKEH0iK8ANJEX4gKcIPJEX4gaQIP5BU13l+2xOSvibpeERcWWx7UNI/SpoudrsvIp6pq0n0r9t343/44YcDHX/FihUDjY/m9HLm3ybppjm2fz8iVhc/BB+YZ7qGPyJelPTeEHoBMESDvOffYHuP7QnbF1bWEYCh6Df8P5T0WUmrJR2R1PGL4Gyvtz1pe3J6errTbgCGrK/wR8SxiDgdEWck/UjSmpJ9t0REOyLarVar3z4BVKyv8NteOevu1yW9UU07AIall6m+xyV9UdIy24ck/bOkL9peLSkkTUn6Vo09AqhB1/BHxLo5Nm+toRf06YUXXuhY27dvX61j33XXXaX1pUuX1jo++scVfkBShB9IivADSRF+ICnCDyRF+IGk+OrueeDUqVOl9U2bNnWsHT16dKCxV61aVVrfuHHjQMdHczjzA0kRfiApwg8kRfiBpAg/kBThB5Ii/EBSzPPPAwcPHiytv/rqq7WNvXVr+V9vn3/++bWNjXpx5geSIvxAUoQfSIrwA0kRfiApwg8kRfiBpJjnHwHdljG7+uqrS+sR0ffY9957b2n9uuuu6/vYGG2c+YGkCD+QFOEHkiL8QFKEH0iK8ANJEX4gqa7z/LYvkbRD0nJJIWlLRPzA9kWSfippTNKUpNsj4vf1tXru2rFjR2n95MmTtY192223ldYXLFhQ29hoVi9n/lOSNkXEFZKukfRt21dIukfSroi4XNKu4j6AeaJr+CPiSES8Utw+KektSRdLWitpe7Hbdkm31tUkgOp9ovf8tsckfU7SryUtj4gjRemoZt4WAJgneg6/7U9J+pmk70TEH2bXYubi8jkvMLe93vak7clu17ADGJ6ewm97oWaC/+OI+Hmx+ZjtlUV9paTjcz02IrZERDsi2q1Wq4qeAVSga/htW9JWSW9FxPdmlXZKGi9uj0t6uvr2ANSllz/p/YKkb0p63fZrxbb7JD0s6Qnbd0o6IOn2elqc/959993S+kMPPVTb2HfccUdp/aqrrqptbIy2ruGPiF9Jcofyl6ptB8CwcIUfkBThB5Ii/EBShB9IivADSRF+ICm+unsIlixZUlq/4IILSuvvv/9+aX3x4sUda92uITjvPP4JZMWZH0iK8ANJEX4gKcIPJEX4gaQIP5AU4QeSYpJ3CLp9fdmBAwdK64sWLSqtv/POOx1rK1asKH0s8uLMDyRF+IGkCD+QFOEHkiL8QFKEH0iK8ANJMc8/BJdddllpfWa1M2C4OPMDSRF+ICnCDyRF+IGkCD+QFOEHkiL8QFJdw2/7Etv/Zfs3tt+0vbHY/qDtw7ZfK35urr9dAFXp5SKfU5I2RcQrtpdKetn2c0Xt+xHxr/W1B6AuXcMfEUckHSlun7T9lqSL624MQL0+0Xt+22OSPifp18WmDbb32J6wfWGHx6y3PWl7stvXWQEYnp7Db/tTkn4m6TsR8QdJP5T0WUmrNfPK4JG5HhcRWyKiHRHtVqtVQcsAqtBT+G0v1EzwfxwRP5ekiDgWEacj4oykH0laU1+bAKrWy6f9lrRV0lsR8b1Z21fO2u3rkt6ovj0Adenl0/4vSPqmpNdtv1Zsu0/SOturJYWkKUnfqqVDALXo5dP+X0nyHKVnqm8HwLBwhR+QFOEHkiL8QFKEH0iK8ANJEX4gKcIPJEX4gaQIP5AU4QeSIvxAUoQfSIrwA0kRfiApD3N5aNvTkg7M2rRM0omhNfDJjGpvo9qXRG/9qrK3v42Inr4vb6jh/9jg9mREtBtroMSo9jaqfUn01q+meuNlP5AU4QeSajr8Wxoev8yo9jaqfUn01q9Gemv0PT+A5jR95gfQkEbCb/sm27+1/bbte5rooRPbU7ZfL1Yenmy4lwnbx22/MWvbRbafs72v+D3nMmkN9TYSKzeXrCzd6HM3aiteD/1lv+0FkvZK+rKkQ5JekrQuIn4z1EY6sD0lqR0Rjc8J2/57SX+UtCMiriy2/Yuk9yLi4eI/zgsj4p9GpLcHJf2x6ZWbiwVlVs5eWVrSrZL+QQ0+dyV93a4GnrcmzvxrJL0dEfsj4k+SfiJpbQN9jLyIeFHSe2dtXitpe3F7u2b+8Qxdh95GQkQciYhXitsnJX20snSjz11JX41oIvwXSzo46/4hjdaS3yHpF7Zftr2+6WbmsLxYNl2Sjkpa3mQzc+i6cvMwnbWy9Mg8d/2seF01PvD7uGsj4vOSvirp28XL25EUM+/ZRmm6pqeVm4dljpWl/6zJ567fFa+r1kT4D0u6ZNb9TxfbRkJEHC5+H5f0pEZv9eFjHy2SWvw+3nA/fzZKKzfPtbK0RuC5G6UVr5sI/0uSLrf9GduLJH1D0s4G+vgY20uKD2Jke4mkr2j0Vh/eKWm8uD0u6ekGe/kLo7Jyc6eVpdXwczdyK15HxNB/JN2smU/8fyfpu0300KGvSyX9T/HzZtO9SXpcMy8D/08zn43cKemvJe2StE/SLyVdNEK9/buk1yXt0UzQVjbU27WaeUm/R9Jrxc/NTT93JX018rxxhR+QFB/4AUkRfiApwg8kRfiBpAg/kBThB5Ii/EBShB9I6v8BrhzvFfpZTAUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'meta': {'puid': 'nn5g30lmmgj5sqg1etdcbi4695', 'requestPath': {'sk-mnist-classifier': 'seldonio/sk-example-mnist:0.2'}}, 'data': {'names': ['class:0', 'class:1', 'class:2', 'class:3', 'class:4', 'class:5', 'class:6', 'class:7', 'class:8', 'class:9'], 'ndarray': [[0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]]}}\n",
      "{\n",
      "  \"class:0\": \"0.00\",\n",
      "  \"class:1\": \"1.00\",\n",
      "  \"class:2\": \"0.00\",\n",
      "  \"class:3\": \"0.00\",\n",
      "  \"class:4\": \"0.00\",\n",
      "  \"class:5\": \"0.00\",\n",
      "  \"class:6\": \"0.00\",\n",
      "  \"class:7\": \"0.00\",\n",
      "  \"class:8\": \"0.00\",\n",
      "  \"class:9\": \"0.00\"\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "predict_rest_mnist(mnist,\"mnist-classifier\",\"seldon\",ISTIO_GATEWAY)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Start a Load Test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "node/minikube labeled\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl label nodes $(kubectl get nodes -o jsonpath='{.items[0].metadata.name}') role=locust"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME:   loadtest\n",
      "LAST DEPLOYED: Sun Jun 30 16:23:57 2019\n",
      "NAMESPACE: seldon\n",
      "STATUS: DEPLOYED\n",
      "\n",
      "RESOURCES:\n",
      "==> v1/Pod(related)\n",
      "NAME                   READY  STATUS       RESTARTS  AGE\n",
      "locust-master-1-mblc6  0/2    Init:0/1     0         1s\n",
      "locust-master-1-trgc5  0/2    Terminating  0         85s\n",
      "locust-slave-1-jgqp2   0/2    Init:0/1     0         1s\n",
      "locust-slave-1-mdgxc   0/2    Terminating  0         85s\n",
      "\n",
      "==> v1/ReplicationController\n",
      "NAME             DESIRED  CURRENT  READY  AGE\n",
      "locust-master-1  1        1        0      1s\n",
      "locust-slave-1   1        1        0      1s\n",
      "\n",
      "==> v1/Service\n",
      "NAME             TYPE      CLUSTER-IP     EXTERNAL-IP  PORT(S)                                       AGE\n",
      "locust-master-1  NodePort  10.98.215.152  <none>       5557:31305/TCP,5558:31110/TCP,8089:31793/TCP  1s\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!helm install ../../../helm-charts/seldon-core-loadtesting --name loadtest  \\\n",
    "    --namespace seldon \\\n",
    "    --repo https://storage.googleapis.com/seldon-charts \\\n",
    "    --set locust.script=mnist_rest_locust.py \\\n",
    "    --set locust.host=http://{ISTIO_GATEWAY} \\\n",
    "    --set rest.pathPrefix=/seldon/seldon/mnist-classifier \\\n",
    "    --set oauth.enabled=false \\\n",
    "    --set oauth.key=oauth-key \\\n",
    "    --set oauth.secret=oauth-secret \\\n",
    "    --set locust.hatchRate=1 \\\n",
    "    --set locust.clients=1 \\\n",
    "    --set loadtest.sendFeedback=1 \\\n",
    "    --set locust.minWait=0 \\\n",
    "    --set locust.maxWait=0 \\\n",
    "    --set replicaCount=1 \\\n",
    "    --set data.size=784\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we will add a canary and split traffic 75% to 25% to it. This is done by adding a new predictor to the SeldonDeployment and specifying the traffic values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\r\n",
      "    \u001b[34;01m\"apiVersion\"\u001b[39;49;00m: \u001b[33m\"machinelearning.seldon.io/v1alpha2\"\u001b[39;49;00m,\r\n",
      "    \u001b[34;01m\"kind\"\u001b[39;49;00m: \u001b[33m\"SeldonDeployment\"\u001b[39;49;00m,\r\n",
      "    \u001b[34;01m\"metadata\"\u001b[39;49;00m: {\r\n",
      "        \u001b[34;01m\"labels\"\u001b[39;49;00m: {\r\n",
      "            \u001b[34;01m\"app\"\u001b[39;49;00m: \u001b[33m\"seldon\"\u001b[39;49;00m\r\n",
      "        },\r\n",
      "        \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"mnist-classifier\"\u001b[39;49;00m\r\n",
      "    },\r\n",
      "    \u001b[34;01m\"spec\"\u001b[39;49;00m: {\r\n",
      "        \u001b[34;01m\"annotations\"\u001b[39;49;00m: {\r\n",
      "            \u001b[34;01m\"project_name\"\u001b[39;49;00m: \u001b[33m\"Mnist classification\"\u001b[39;49;00m\r\n",
      "        },\r\n",
      "        \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"mnist-deployment\"\u001b[39;49;00m,\r\n",
      "        \u001b[34;01m\"predictors\"\u001b[39;49;00m: [\r\n",
      "            {\r\n",
      "                \u001b[34;01m\"componentSpecs\"\u001b[39;49;00m: [{\r\n",
      "                    \u001b[34;01m\"spec\"\u001b[39;49;00m: {\r\n",
      "                        \u001b[34;01m\"containers\"\u001b[39;49;00m: [\r\n",
      "                            {\r\n",
      "                                \u001b[34;01m\"image\"\u001b[39;49;00m: \u001b[33m\"seldonio/sk-example-mnist:0.2\"\u001b[39;49;00m,\r\n",
      "                                \u001b[34;01m\"imagePullPolicy\"\u001b[39;49;00m: \u001b[33m\"IfNotPresent\"\u001b[39;49;00m,\r\n",
      "                                \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"sk-mnist-classifier\"\u001b[39;49;00m,\r\n",
      "                                \u001b[34;01m\"resources\"\u001b[39;49;00m: {\r\n",
      "                                    \u001b[34;01m\"requests\"\u001b[39;49;00m: {\r\n",
      "                                        \u001b[34;01m\"memory\"\u001b[39;49;00m: \u001b[33m\"1Mi\"\u001b[39;49;00m\r\n",
      "                                    }\r\n",
      "                                }\r\n",
      "                            }\r\n",
      "                        ],\r\n",
      "                        \u001b[34;01m\"terminationGracePeriodSeconds\"\u001b[39;49;00m: \u001b[34m1\u001b[39;49;00m\r\n",
      "                    }\r\n",
      "                }],\r\n",
      "                \u001b[34;01m\"graph\"\u001b[39;49;00m: {\r\n",
      "                    \u001b[34;01m\"children\"\u001b[39;49;00m: [],\r\n",
      "                    \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"sk-mnist-classifier\"\u001b[39;49;00m,\r\n",
      "                    \u001b[34;01m\"endpoint\"\u001b[39;49;00m: {\r\n",
      "                        \u001b[34;01m\"type\"\u001b[39;49;00m : \u001b[33m\"REST\"\u001b[39;49;00m\r\n",
      "                    },\r\n",
      "                    \u001b[34;01m\"type\"\u001b[39;49;00m: \u001b[33m\"MODEL\"\u001b[39;49;00m\r\n",
      "                },\r\n",
      "                \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"sk-mnist-predictor\"\u001b[39;49;00m,\r\n",
      "                \u001b[34;01m\"replicas\"\u001b[39;49;00m: \u001b[34m1\u001b[39;49;00m,\r\n",
      "\t\t\u001b[34;01m\"labels\"\u001b[39;49;00m:{\r\n",
      "\t\t    \u001b[34;01m\"version\"\u001b[39;49;00m:\u001b[33m\"v1\"\u001b[39;49;00m\r\n",
      "\t\t},\r\n",
      "\t\t\u001b[34;01m\"traffic\"\u001b[39;49;00m: \u001b[34m75\u001b[39;49;00m\r\n",
      "            },\r\n",
      "            {\r\n",
      "                \u001b[34;01m\"componentSpecs\"\u001b[39;49;00m: [{\r\n",
      "                    \u001b[34;01m\"spec\"\u001b[39;49;00m: {\r\n",
      "                        \u001b[34;01m\"containers\"\u001b[39;49;00m: [\r\n",
      "                            {\r\n",
      "                                \u001b[34;01m\"image\"\u001b[39;49;00m: \u001b[33m\"seldonio/tf-example-mnist:0.1\"\u001b[39;49;00m,\r\n",
      "                                \u001b[34;01m\"imagePullPolicy\"\u001b[39;49;00m: \u001b[33m\"IfNotPresent\"\u001b[39;49;00m,\r\n",
      "                                \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"tf-mnist-classifier\"\u001b[39;49;00m,\r\n",
      "                                \u001b[34;01m\"resources\"\u001b[39;49;00m: {\r\n",
      "                                    \u001b[34;01m\"requests\"\u001b[39;49;00m: {\r\n",
      "                                        \u001b[34;01m\"memory\"\u001b[39;49;00m: \u001b[33m\"1Mi\"\u001b[39;49;00m\r\n",
      "                                    }\r\n",
      "                                }\r\n",
      "                            }\r\n",
      "                        ],\r\n",
      "                        \u001b[34;01m\"terminationGracePeriodSeconds\"\u001b[39;49;00m: \u001b[34m1\u001b[39;49;00m\r\n",
      "                    }\r\n",
      "                }],\r\n",
      "                \u001b[34;01m\"graph\"\u001b[39;49;00m: {\r\n",
      "                    \u001b[34;01m\"children\"\u001b[39;49;00m: [],\r\n",
      "                    \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"tf-mnist-classifier\"\u001b[39;49;00m,\r\n",
      "                    \u001b[34;01m\"endpoint\"\u001b[39;49;00m: {\r\n",
      "                        \u001b[34;01m\"type\"\u001b[39;49;00m : \u001b[33m\"REST\"\u001b[39;49;00m\r\n",
      "                    },\r\n",
      "                    \u001b[34;01m\"type\"\u001b[39;49;00m: \u001b[33m\"MODEL\"\u001b[39;49;00m\r\n",
      "                },\r\n",
      "                \u001b[34;01m\"name\"\u001b[39;49;00m: \u001b[33m\"tf-mnist-predictor\"\u001b[39;49;00m,\r\n",
      "                \u001b[34;01m\"replicas\"\u001b[39;49;00m: \u001b[34m1\u001b[39;49;00m,\r\n",
      "\t\t\u001b[34;01m\"labels\"\u001b[39;49;00m:{\r\n",
      "\t\t    \u001b[34;01m\"version\"\u001b[39;49;00m:\u001b[33m\"v2\"\u001b[39;49;00m\r\n",
      "\t\t},\r\n",
      "\t\t\u001b[34;01m\"traffic\"\u001b[39;49;00m: \u001b[34m25\u001b[39;49;00m\r\n",
      "            }\r\n",
      "\t    \r\n",
      "        ]\r\n",
      "    }\r\n",
      "}\r\n"
     ]
    }
   ],
   "source": [
    "!pygmentize mnist_v2.json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.38.0 (20140413.2041)\n",
       " -->\n",
       "<!-- Title: %3 Pages: 1 -->\n",
       "<svg width=\"296pt\" height=\"171pt\"\n",
       " viewBox=\"0.00 0.00 296.00 171.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 167)\">\n",
       "<title>%3</title>\n",
       "<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-167 292,-167 292,4 -4,4\"/>\n",
       "<g id=\"clust1\" class=\"cluster\"><title>cluster_0</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"8,-8 8,-155 142,-155 142,-8 8,-8\"/>\n",
       "<text text-anchor=\"middle\" x=\"75\" y=\"-139.8\" font-family=\"Times,serif\" font-size=\"14.00\">predictor</text>\n",
       "</g>\n",
       "<g id=\"clust2\" class=\"cluster\"><title>cluster_1</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"150,-8 150,-155 280,-155 280,-8 150,-8\"/>\n",
       "<text text-anchor=\"middle\" x=\"215\" y=\"-139.8\" font-family=\"Times,serif\" font-size=\"14.00\">canary</text>\n",
       "</g>\n",
       "<!-- sk&#45;mnist&#45;classifier0 -->\n",
       "<g id=\"node1\" class=\"node\"><title>sk&#45;mnist&#45;classifier0</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"133.5,-124 16.5,-124 16.5,-88 133.5,-88 133.5,-124\"/>\n",
       "<text text-anchor=\"middle\" x=\"75\" y=\"-102.3\" font-family=\"Times,serif\" font-size=\"14.00\">sk&#45;mnist&#45;classifier</text>\n",
       "</g>\n",
       "<!-- sk&#45;mnist&#45;classifier0endpoint -->\n",
       "<g id=\"node2\" class=\"node\"><title>sk&#45;mnist&#45;classifier0endpoint</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"75\" cy=\"-34\" rx=\"32.4942\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"75\" y=\"-30.3\" font-family=\"Times,serif\" font-size=\"14.00\">REST</text>\n",
       "</g>\n",
       "<!-- sk&#45;mnist&#45;classifier0&#45;&gt;sk&#45;mnist&#45;classifier0endpoint -->\n",
       "<g id=\"edge1\" class=\"edge\"><title>sk&#45;mnist&#45;classifier0&#45;&gt;sk&#45;mnist&#45;classifier0endpoint</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M75,-87.6966C75,-79.9827 75,-70.7125 75,-62.1124\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"78.5001,-62.1043 75,-52.1043 71.5001,-62.1044 78.5001,-62.1043\"/>\n",
       "</g>\n",
       "<!-- tf&#45;mnist&#45;classifier1 -->\n",
       "<g id=\"node3\" class=\"node\"><title>tf&#45;mnist&#45;classifier1</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"271.5,-124 158.5,-124 158.5,-88 271.5,-88 271.5,-124\"/>\n",
       "<text text-anchor=\"middle\" x=\"215\" y=\"-102.3\" font-family=\"Times,serif\" font-size=\"14.00\">tf&#45;mnist&#45;classifier</text>\n",
       "</g>\n",
       "<!-- tf&#45;mnist&#45;classifier1endpoint -->\n",
       "<g id=\"node4\" class=\"node\"><title>tf&#45;mnist&#45;classifier1endpoint</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"215\" cy=\"-34\" rx=\"32.4942\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"215\" y=\"-30.3\" font-family=\"Times,serif\" font-size=\"14.00\">REST</text>\n",
       "</g>\n",
       "<!-- tf&#45;mnist&#45;classifier1&#45;&gt;tf&#45;mnist&#45;classifier1endpoint -->\n",
       "<g id=\"edge2\" class=\"edge\"><title>tf&#45;mnist&#45;classifier1&#45;&gt;tf&#45;mnist&#45;classifier1endpoint</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M215,-87.6966C215,-79.9827 215,-70.7125 215,-62.1124\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"218.5,-62.1043 215,-52.1043 211.5,-62.1044 218.5,-62.1043\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.dot.Digraph at 0x7f28fc35acc0>"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_graph(\"mnist_v2.json\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io/mnist-classifier configured\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply -f mnist_v2.json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAADnZJREFUeJzt3XGMlPWdx/HP94SqATRY1oUA3vYaIRKSg8uEmIDKpbeNJY1QSBRMGjCkayIaK43RuCY1MTHmctKYeKmhJ4KXHq0GkMWQa5U0KqExjMoB4nF6ZisQhAUM0D+0Ct/+sQ/Nqju/GWaemWeW7/uVbHbm+T7PPN887Idn5nnmeX7m7gIQz98V3QCAYhB+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBjWrlyiZMmOBdXV2tXCUQSn9/v06cOGG1zNtQ+M3sVklPS7pM0n+4+5Op+bu6ulQulxtZJYCEUqlU87x1v+03s8sk/bukH0iaIWmZmc2o9/UAtFYjn/nnSPrQ3T9y979I+o2khfm0BaDZGgn/ZEmHhjw/nE37CjPrMbOymZUHBgYaWB2APDX9aL+7r3X3kruXOjo6mr06ADVqJPxHJE0d8nxKNg3ACNBI+HdLut7MvmNm35K0VFJfPm0BaLa6T/W5+5dmdq+k32nwVN86d38vt84ANFVD5/ndfbuk7Tn1AqCF+HovEBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IivADQTU0Sq+Z9Us6K+mcpC/dvZRHU/iq8+fPJ+v9/f11v/b69euT9TNnziTrH3/8cbK+ZcuWirW5c+cml3300UeT9fnz5yfrV1xxRbIeXUPhz/yzu5/I4XUAtBBv+4GgGg2/S/q9mb1tZj15NASgNRp92z/P3Y+Y2bWSXjWz/3X3N4bOkP2n0CNJ1113XYOrA5CXhvb87n4k+31c0hZJc4aZZ627l9y91NHR0cjqAOSo7vCb2RgzG3fhsaTvS9qfV2MAmquRt/2dkraY2YXX+S93/+9cugLQdHWH390/kvSPOfYyYu3bty9Zr3a+uppz584l69u3b2/o9VPcPVnP/vOvq75r167ksgsWLEjWe3t7k/XHH388WY+OU31AUIQfCIrwA0ERfiAowg8ERfiBoPK4qi+8jRs3Juvbtm1r6vrHjx9fsbZkyZKmrnvKlCnJ+uLFiyvW1qxZk1y22uXGzzzzTLLe3d1dsXbzzTcnl42APT8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBMV5/hzMmzcvWe/s7EzWp06dmqw/8cQTyXrqFthXXnllctlmO3nyZMXaoUOHGnrtzz77LFmfNm1aQ69/qWPPDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBcZ4/B9VuMX306NFk/eDBg8n69OnTL7qnvFQb/nv37t3Jek9P5SEcT58+nVx23LhxyXq16/0nTpyYrEfHnh8IivADQRF+ICjCDwRF+IGgCD8QFOEHgqp6nt/M1kn6oaTj7j4zm3aNpN9K6pLUL+l2d/+0eW1e2pp5Hv/w4cPJ+sqVK5P1vXv3JuvHjh276J4uWLFiRbK+evXqZH3mzJl1rxu17fnXS7r1a9MelrTD3a+XtCN7DmAEqRp+d39D0qmvTV4oaUP2eIOkRTn3BaDJ6v3M3+nuF76z+omk9H2qALSdhg/4ubtL8kp1M+sxs7KZlQcGBhpdHYCc1Bv+Y2Y2SZKy38crzejua9295O6ljo6OOlcHIG/1hr9P0vLs8XJJW/NpB0CrVA2/mW2U9EdJ083ssJmtlPSkpG4z+0DSv2TPAYwgVc/zu/uyCqXv5dwL6pQ6F3/LLbckl612TX21a+J7e3uT9RkzZlSsLV26NLmsmSXraAzf8AOCIvxAUIQfCIrwA0ERfiAowg8Exa27R4BXXnklWX/wwQcr1ga/fV3ZU089law/8MADyTpGLvb8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU5/lHgLNnzybrqSG+q909admySlds41LHnh8IivADQRF+ICjCDwRF+IGgCD8QFOEHguI8/wiwePHiZD01VPX+/fuTy959993J+tatjMdyqWLPDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBVT3Pb2brJP1Q0nF3n5lNe0zSTyQNZLM94u7bm9VkdJdffnmy/tJLL1WszZ8/P7lsX19fsn7bbbcl6xs3bkzWx4wZk6yjOLXs+ddLunWY6b9w91nZD8EHRpiq4Xf3NySdakEvAFqokc/895rZXjNbZ2bjc+sIQEvUG/5fSvqupFmSjkqqOOCbmfWYWdnMygMDA5VmA9BidYXf3Y+5+zl3Py/pV5LmJOZd6+4ldy9Vu5kkgNapK/xmNmnI0x9JSl86BqDt1HKqb6Ok+ZImmNlhST+XNN/MZklySf2S0teFAmg7VcPv7sPd2P25JvSCOk2fPr1i7d13300uO3v27GR927ZtyfqKFSuS9eeff75ibezYscll0Vx8ww8IivADQRF+ICjCDwRF+IGgCD8QFLfuvsRNnDgxWX/99deT9SVLliTrmzZtStbPnTtXsfbiiy8mlx01ij/PZmLPDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBcSI1uGnTpiXra9asSdbvv//+ZP3ll1+uWNu+PX3T52q3DUdj2PMDQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCc52+BN998M1nfsWNHst7b25usjx49+qJ7qlV3d3eyft999yXrq1atqli76667kssePHgwWZ8wYUKyjjT2/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QVNXz/GY2VdILkjoluaS17v60mV0j6beSuiT1S7rd3T9tXqvt69SpU8n64sWLk/WTJ08m6/fcc0+yfu211ybrzXTTTTcl61dffXXF2qefpv9cXnvttWR96dKlyTrSatnzfynpZ+4+Q9KNklaZ2QxJD0va4e7XS9qRPQcwQlQNv7sfdfd3ssdnJb0vabKkhZI2ZLNtkLSoWU0CyN9FfeY3sy5JsyW9JanT3Y9mpU80+LEAwAhRc/jNbKykTZJ+6u5nhtbc3TV4PGC45XrMrGxm5YGBgYaaBZCfmsJvZqM1GPxfu/vmbPIxM5uU1SdJOj7csu6+1t1L7l7q6OjIo2cAOagafjMzSc9Jet/dh97KtU/S8uzxcklb828PQLPUcknvXEk/lrTPzPZk0x6R9KSkF81spaQ/Sbq9OS22vy+++CJZr3Yqr5pdu3Yl64sWFXes9YYbbkjWJ0+eXLF2+vTpvNvBRagafnffKckqlL+XbzsAWoVv+AFBEX4gKMIPBEX4gaAIPxAU4QeC4tbdObjqqquS9QULFiTr1YaqvuOOO5L11atXV6zdeeedyWUb9dZbbyXrBw4cqFgbNSr955e6HBiNY88PBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0HZ4B24WqNUKnm5XG7Z+tpFtevWH3rooWR9w4YNyfrnn39+0T3Vqtrfx+C9Xupz4403JuvV7mOAbyqVSiqXyzX9o7DnB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGguJ6/Bapdl/7ss88m693d3cn6zp07K9Y2b95csSZJhw4dStYbNXPmzIq1vr6+pq4baez5gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiCoqtfzm9lUSS9I6pTkkta6+9Nm9pikn0gayGZ9xN2TN6CPej0/0CoXcz1/LV/y+VLSz9z9HTMbJ+ltM3s1q/3C3f+t3kYBFKdq+N39qKSj2eOzZva+pMnNbgxAc13UZ34z65I0W9KFMZruNbO9ZrbOzMZXWKbHzMpmVh4YGBhuFgAFqDn8ZjZW0iZJP3X3M5J+Kem7kmZp8J3BU8Mt5+5r3b3k7qWOjo4cWgaQh5rCb2ajNRj8X7v7Zkly92Pufs7dz0v6laQ5zWsTQN6qht8Gb8/6nKT33X3NkOmThsz2I0n7828PQLPUcrR/rqQfS9pnZnuyaY9IWmZmszR4+q9f0t1N6RBAU9RytH+npOHOG6YHlQfQ1viGHxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IivADQRF+IKiqt+7OdWVmA5L+NGTSBEknWtbAxWnX3tq1L4ne6pVnb3/v7jXdL6+l4f/Gys3K7l4qrIGEdu2tXfuS6K1eRfXG234gKMIPBFV0+NcWvP6Udu2tXfuS6K1ehfRW6Gd+AMUpes8PoCCFhN/MbjWzg2b2oZk9XEQPlZhZv5ntM7M9ZlbokMLZMGjHzWz/kGnXmNmrZvZB9nvYYdIK6u0xMzuSbbs9ZragoN6mmtkfzOyAmb1nZvdn0wvddom+CtluLX/bb2aXSfo/Sd2SDkvaLWmZux9oaSMVmFm/pJK7F35O2MxulvRnSS+4+8xs2r9KOuXuT2b/cY5394fapLfHJP256JGbswFlJg0dWVrSIkkrVOC2S/R1uwrYbkXs+edI+tDdP3L3v0j6jaSFBfTR9tz9DUmnvjZ5oaQN2eMNGvzjabkKvbUFdz/q7u9kj89KujCydKHbLtFXIYoI/2RJh4Y8P6z2GvLbJf3ezN42s56imxlGZzZsuiR9IqmzyGaGUXXk5lb62sjSbbPt6hnxOm8c8Pumee7+T5J+IGlV9va2LfngZ7Z2Ol1T08jNrTLMyNJ/U+S2q3fE67wVEf4jkqYOeT4lm9YW3P1I9vu4pC1qv9GHj10YJDX7fbzgfv6mnUZuHm5kabXBtmunEa+LCP9uSdeb2XfM7FuSlkrqK6CPbzCzMdmBGJnZGEnfV/uNPtwnaXn2eLmkrQX28hXtMnJzpZGlVfC2a7sRr9295T+SFmjwiP//S+otoocKff2DpP/Jft4rujdJGzX4NvALDR4bWSnp25J2SPpA0muSrmmj3v5T0j5JezUYtEkF9TZPg2/p90rak/0sKHrbJfoqZLvxDT8gKA74AUERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8I6q9+ZHLJOvkaFQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'meta': {'puid': 'pii2cg6n9gpke7gag6etkuj4cv', 'requestPath': {'sk-mnist-classifier': 'seldonio/sk-example-mnist:0.2'}}, 'data': {'names': ['class:0', 'class:1', 'class:2', 'class:3', 'class:4', 'class:5', 'class:6', 'class:7', 'class:8', 'class:9'], 'ndarray': [[0.0, 0.0, 0.03333333333333333, 0.8666666666666667, 0.0, 0.06666666666666667, 0.0, 0.0, 0.03333333333333333, 0.0]]}}\n",
      "{\n",
      "  \"class:0\": \"0.00\",\n",
      "  \"class:1\": \"0.00\",\n",
      "  \"class:2\": \"0.03\",\n",
      "  \"class:3\": \"0.87\",\n",
      "  \"class:4\": \"0.00\",\n",
      "  \"class:5\": \"0.07\",\n",
      "  \"class:6\": \"0.00\",\n",
      "  \"class:7\": \"0.00\",\n",
      "  \"class:8\": \"0.03\",\n",
      "  \"class:9\": \"0.00\"\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "predict_rest_mnist(mnist,\"mnist-classifier\",\"seldon\",ISTIO_GATEWAY)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You should see traffic being split on the Istio service dashboard for the mnist-classifier.\n",
    "\n",
    "![skpredictor](sk-predictor.png)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![tfpredictor](tf-predictor.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When you are happy the canary is ok you can promote to full traffic."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io/mnist-classifier configured\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply -f mnist_v3.json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAADm5JREFUeJzt3X+MVfWZx/HPI0uDQmPUmUWk4FgkG5XAsLngxhLT1YVQQxxQgyWxstFIjZiI9o9V9w9NjIlZbUmNmybDQkoNC0ioAY2uVdAYYm24GqtY3cUFKhBkZuKPUv2jizz7xxyaEed873jvufdc5nm/ksnce57zvefJzXzm3HvOvedr7i4A8ZxRdgMAykH4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8E9Tet3FhHR4d3dXW1cpNAKAcOHNDAwICNZN2Gwm9mCyX9XNIYSf/h7o+k1u/q6lK1Wm1kkwASKpXKiNet+2W/mY2R9O+SfiDpUknLzOzSeh8PQGs18p5/rqQP3H2fu/9F0iZJPcW0BaDZGgn/ZEkHh9w/lC37CjNbYWZVM6v29/c3sDkARWr60X5373X3irtXOjs7m705ACPUSPgPS5oy5P53smUATgONhH+3pOlmdpGZfUvSDyVtL6YtAM1W96k+dz9uZndKekGDp/rWufu7hXUGoKkaOs/v7s9Jeq6gXgC0EB/vBYIi/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiColk7Rjfp8/vnnyfq1116bW9u5c2dy7EUXXZSsX3755cn6BRdckKxff/31ubUrrrgiORbNxZ4fCIrwA0ERfiAowg8ERfiBoAg/EBThB4Jq6Dy/mR2QdEzSl5KOu3uliKbwVY8//niyXutcfsr+/fsbqteS6v3mm29Ojl27dm1D20ZaER/y+Ud3HyjgcQC0EC/7gaAaDb9L+o2ZvWFmK4poCEBrNPqyf567Hzazv5X0opm97+6vDl0h+6ewQpKmTp3a4OYAFKWhPb+7H85+90l6WtLcYdbpdfeKu1c6Ozsb2RyAAtUdfjMbb2bfPnlb0gJJe4pqDEBzNfKyf6Kkp83s5OP8p7v/VyFdAWi6usPv7vskzSqwF+TYvXt3st7R0ZFb27hxY3Ls8ePHk/Xzzz8/WX/ttdeS9TVr1uTWnnzyyeTYhx9+OFmv1RvSONUHBEX4gaAIPxAU4QeCIvxAUIQfCIpLd58Gap3SGhjI/1Jlb29vcuxTTz1VV08ndXd3J+u33HJLbm3p0qXJsbNnz07WX3/99WT9wgsvTNajY88PBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0Fxnr8NHD16NFnfunVr3Y999dVX1z22COPGjcut7dq1Kzn2k08+SdZrfR0Zaez5gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAozvO3gbPPPruhejZ3wrBuuummunoqSup6AseOHWvosT/88MNkfdq0aQ09/mjHnh8IivADQRF+ICjCDwRF+IGgCD8QFOEHgqp5nt/M1klaJKnP3Wdky86VtFlSl6QDkpa6e/rL18iV+s67JM2alZ4JPXX9+vHjxyfH1vrO/EMPPZSsT548OVl/7LHHcmuNTg8+c+bMZB1pI9nz/1LSwlOW3Stph7tPl7Qjuw/gNFIz/O7+qqSPT1ncI2l9dnu9pMUF9wWgyep9zz/R3Y9ktz+SNLGgfgC0SMMH/NzdJXle3cxWmFnVzKr9/f2Nbg5AQeoN/1EzmyRJ2e++vBXdvdfdK+5e6ezsrHNzAIpWb/i3S1qe3V4uaVsx7QBolZrhN7ONkn4r6e/M7JCZ3SrpEUnzzWyvpH/K7gM4jdQ8z+/uy3JK5V4QPpA5c+Yk688++2xurdY1/wcGBpL11atXJ+uNmDBhQrK+YcOGZP28884rsp1w+IQfEBThB4Ii/EBQhB8IivADQRF+ICgu3X0auOGGG5L1++67L7d2zz33JMd+8cUXdfVUhC1btiTrV111VYs6iYk9PxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ExXn+00CtKyB1dXXl1vbt21dwN8XZuXNnsr5w4akXjUaR2PMDQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCc5z8NPPHEE8l6mefye3p6kvVnnnkmt/boo48mx9aa/vuuu+5K1pHGnh8IivADQRF+ICjCDwRF+IGgCD8QFOEHgjJ3T69gtk7SIkl97j4jW/agpNsk9Wer3e/uz9XaWKVS8Wq12lDDo9GePXuS9VmzZiXrJ06cyK1NmjQpOfbIkSPJei21/n42b96cW6s1ffgrr7ySrL/00kvJ+syZM5P10ahSqahardpI1h3Jnv+Xkoa7qsJqd+/OfmoGH0B7qRl+d39V0sct6AVACzXynv9OM3vbzNaZ2TmFdQSgJeoN/y8kTZPULemIpJ/mrWhmK8ysambV/v7+vNUAtFhd4Xf3o+7+pbufkLRG0tzEur3uXnH3Sq0LUQJonbrCb2ZDDyEvkZQ+XA2g7dT8Sq+ZbZT0fUkdZnZI0gOSvm9m3ZJc0gFJP25ijwCaoGb43X3ZMIvXNqGXsPr6+pL11Hl8SRo3blxubdu2bcmxF198cbLeqBtvvDG3dt111yXHTp06te7HlqTUZ0rGjx+fHBsBn/ADgiL8QFCEHwiK8ANBEX4gKMIPBMWlu9vAyy+/3ND4jo6O3NqcOXMaeuxmGjt2bLI+ZsyYZP39999P1vfv359bmzFjRnJsBOz5gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAozvOPAlOmTCm7haaodentw4cPJ+uffvppke2MOuz5gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAozvO3gUsuuaSh8QcPHiyok/ZS69Ldtbzwwgu5tXnz5jX02KMBe34gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCKrmeX4zmyLpV5ImSnJJve7+czM7V9JmSV2SDkha6u6fNK/V0WvJkiXJeq1ptAcGBnJr27dvT45dsGBBsp6a/rtRe/fuTdY3bdrU0OPPnz+/ofGj3Uj2/Mcl/cTdL5X0D5JWmtmlku6VtMPdp0vakd0HcJqoGX53P+Lub2a3j0l6T9JkST2S1merrZe0uFlNAijeN3rPb2ZdkmZL+p2kie5+JCt9pMG3BQBOEyMOv5lNkLRV0ip3/9PQmru7Bo8HDDduhZlVzaza39/fULMAijOi8JvZWA0Gf4O7/zpbfNTMJmX1SZL6hhvr7r3uXnH3SmdnZxE9AyhAzfCbmUlaK+k9d//ZkNJ2Scuz28slbSu+PQDNMpKv9H5P0o8kvWNmb2XL7pf0iKSnzOxWSX+UtLQ5LY5+Z555ZrJ+9913J+urVq3KrfX09CTHdnd3J+u33XZbsj558uRkffr06bm1O+64Izn2s88+S9avvPLKZL3WKdLoaobf3XdJspzy1cW2A6BV+IQfEBThB4Ii/EBQhB8IivADQRF+ICgu3X0aqHU+/Iwz8v+Hpz4DIEl79uxJ1leuXJmsN9Nll12WrD///PPJ+llnnVVkO6MOe34gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrz/KPA7bffnltbtGhRcuzOnTuT9bVr1ybrtb7Pv2XLltza4sXpa74+8MADyTrn8RvDnh8IivADQRF+ICjCDwRF+IGgCD8QFOEHgrLBmbZao1KpeLVabdn2gGgqlYqq1Wrepfa/gj0/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRVM/xmNsXMXjazP5jZu2Z2V7b8QTM7bGZvZT/XNL9dAEUZycU8jkv6ibu/aWbflvSGmb2Y1Va7+2PNaw9As9QMv7sfkXQku33MzN6TlL58C4C2943e85tZl6TZkn6XLbrTzN42s3Vmdk7OmBVmVjWzan9/f0PNAijOiMNvZhMkbZW0yt3/JOkXkqZJ6tbgK4OfDjfO3XvdveLulc7OzgJaBlCEEYXfzMZqMPgb3P3XkuTuR939S3c/IWmNpLnNaxNA0UZytN8krZX0nrv/bMjySUNWWyIpPd0rgLYykqP935P0I0nvmNlb2bL7JS0zs25JLumApB83pUMATTGSo/27JA33/eDnim8HQKvwCT8gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQLZ2i28z6Jf1xyKIOSQMta+Cbadfe2rUvid7qVWRvF7r7iK6X19Lwf23jZlV3r5TWQEK79taufUn0Vq+yeuNlPxAU4QeCKjv8vSVvP6Vde2vXviR6q1cpvZX6nh9Aecre8wMoSSnhN7OFZvbfZvaBmd1bRg95zOyAmb2TzTxcLbmXdWbWZ2Z7hiw718xeNLO92e9hp0krqbe2mLk5MbN0qc9du8143fKX/WY2RtL/SJov6ZCk3ZKWufsfWtpIDjM7IKni7qWfEzazKyX9WdKv3H1GtuzfJH3s7o9k/zjPcfd/aZPeHpT057Jnbs4mlJk0dGZpSYsl/bNKfO4SfS1VCc9bGXv+uZI+cPd97v4XSZsk9ZTQR9tz91clfXzK4h5J67Pb6zX4x9NyOb21BXc/4u5vZrePSTo5s3Spz12ir1KUEf7Jkg4OuX9I7TXlt0v6jZm9YWYrym5mGBOzadMl6SNJE8tsZhg1Z25upVNmlm6b566eGa+LxgG/r5vn7n8v6QeSVmYvb9uSD75na6fTNSOaublVhplZ+q/KfO7qnfG6aGWE/7CkKUPufydb1hbc/XD2u0/S02q/2YePnpwkNfvdV3I/f9VOMzcPN7O02uC5a6cZr8sI/25J083sIjP7lqQfStpeQh9fY2bjswMxMrPxkhao/WYf3i5peXZ7uaRtJfbyFe0yc3PezNIq+blruxmv3b3lP5Ku0eAR//+V9K9l9JDT13cl/T77ebfs3iRt1ODLwP/T4LGRWyWdJ2mHpL2SXpJ0bhv19qSkdyS9rcGgTSqpt3kafEn/tqS3sp9ryn7uEn2V8rzxCT8gKA74AUERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8I6v8BxjhelW2YyfgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<Response [200]>\n",
      "Route:{}\n",
      "{\n",
      "  \"class:0\": \"0.00\",\n",
      "  \"class:1\": \"0.00\",\n",
      "  \"class:2\": \"0.01\",\n",
      "  \"class:3\": \"0.11\",\n",
      "  \"class:4\": \"0.00\",\n",
      "  \"class:5\": \"0.04\",\n",
      "  \"class:6\": \"0.00\",\n",
      "  \"class:7\": \"0.00\",\n",
      "  \"class:8\": \"0.84\",\n",
      "  \"class:9\": \"0.00\"\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "predict_rest_mnist(mnist,\"mnist-classifier\",\"seldon\",ISTIO_GATEWAY)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You should now see all traffic transfer to the canary.\n",
    "\n",
    "![sk-preditor2](sk-predictor2.png)\n",
    "\n",
    "![tf-preditor2](tf-predictor2.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
